Skip to content

fix(webapp): honor RevokedApiKey grace window for public access tokens#3464

Merged
ericallam merged 1 commit intomainfrom
fix/api-key-rotation-pat-grace-period
Apr 29, 2026
Merged

fix(webapp): honor RevokedApiKey grace window for public access tokens#3464
ericallam merged 1 commit intomainfrom
fix/api-key-rotation-pat-grace-period

Conversation

@ericallam
Copy link
Copy Markdown
Member

Summary

Follow-up to #3420. PATs (public access tokens) minted before an API key rotation 401'd immediately on the realtime stream endpoints, even though the rotation flow advertises a 24h overlap. This fixes the gap.

Root cause

PATs are JWTs signed with the env's apiKey at mint time. When that secret is rotated, validatePublicJwtKey (apps/webapp/app/services/realtime/jwtAuth.server.ts) only verifies the signature against environment.parentEnvironment?.apiKey ?? environment.apiKey — i.e. the env's current canonical key. Any PAT in the wild signed with the previous key fails signature verification → 401, even within the grace window.

#3420 wired up the grace-window fallback in two places — findEnvironmentByApiKey (raw secret-key auth) and api.v1.auth.jwt.ts (signs new JWTs with the canonical key when minting from an old one) — but the verify path for already-issued PATs was never updated.

In a typical app, POST /api/v1/tasks/.../trigger (Bearer secret) keeps working through rotation because that path has the fallback, but GET /realtime/v1/streams/run_*/... and POST /realtime/v1/streams/run_*/input/... 401 for runs that were already in flight when the rotation happened.

Fix

After the primary validateJWT against the env's current apiKey, fall back to non-expired RevokedApiKey rows for the signing env (parent env when the request is against a child) — but only on the failure path, so the hot success path is unchanged. Uses $replica to match the rest of the auth path.

Symmetrical to the findEnvironmentByApiKey two-step from #3420.

Changes

  • apps/webapp/app/services/realtime/jwtAuth.server.tsvalidateAgainstRevokedApiKeys helper invoked only on !result.ok
  • apps/webapp/app/models/runtimeEnvironment.server.tsfindEnvironmentById also selects parentEnvironment.id so we can scope the revoked-keys lookup to the correct env

Test plan

E2E verified locally via curl against GET /realtime/v1/runs/{runId} (PAT-authenticated):

  • Pre-rotation, PAT signed with K1 → 200 with run body
  • Simulate rotation (insert RevokedApiKey row + flip env apiKey to K2 in a single transaction, mirroring regenerateApiKey)
  • Same PAT (K1) within grace window → 200 with run body — fallback hits
  • Fresh PAT signed with K2 → 200 — current key still works
  • Set RevokedApiKey.expiresAt to past → 401 — fallback finds no live row
  • Bogus signature (no rotation) → 401
  • Cleanup verified: env apiKey restored, RevokedApiKey row deleted
  • pnpm run typecheck --filter webapp passes

PATs (public access tokens) are signed with the env's apiKey at mint
time. The original grace-period PR (#3420) updated findEnvironmentByApiKey
to fall back to RevokedApiKey rows for raw secret-key auth, and updated
JWT minting to use the env's current canonical key — but left
validatePublicJwtKey only checking the env's current apiKey. Result:
PATs minted before rotation 401'd immediately on the realtime stream
endpoints, even though the rotation flow advertises a 24h overlap.

Fall back to non-expired RevokedApiKey rows for the signing env (parent
env when the request is against a child) only when the primary signature
check fails. Uses $replica.
@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented Apr 28, 2026

⚠️ No Changeset found

Latest commit: 7a6c83e

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Apr 28, 2026

Walkthrough

This pull request introduces an API key rotation grace period feature. A changelog entry documents the new behavior. The findEnvironmentById function is updated to include the parent environment's id alongside its apiKey in the return type. The core change adds fallback validation logic to validatePublicJwtKey that, when initial JWT validation fails, queries non-expired revoked API keys and attempts re-validation against each one, allowing PATs minted before key rotation to remain functional during the 24-hour grace period.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~28 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and specifically describes the main change: fixing behavior for public access tokens to honor the RevokedApiKey grace window during API key rotation.
Description check ✅ Passed The description provides comprehensive context including root cause, fix details, affected files, and test plan, though it lacks explicit checklist completion and missing Testing section per template.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/api-key-rotation-pat-grace-period

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@ericallam ericallam marked this pull request as ready for review April 28, 2026 17:10
Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 4 additional findings.

Open in Devin Review

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
apps/webapp/app/services/realtime/jwtAuth.server.ts (1)

47-107: ⚠️ Potential issue | 🟠 Major

Scope the revoked-key fallback to signature verification failures only.

This currently falls back on every JWT validation failure, not just signature mismatches. The inline comment at line 44 states "signature verification fails" but the code at line 47 triggers on any !result.ok, including ERR_JWT_EXPIRED and ERR_JWT_CLAIM_INVALID. This wastes replica reads on expired or claim-invalid tokens that don't come from revoked keys.

Additionally, validateAgainstRevokedApiKeys() only returns when the fallback validation succeeds (line 102), so if a revoked key signed a token that is now expired or claim-invalid, the original error is returned, masking the fact that the signature matched a revoked key.

Gate the fallback on signature-verification error code(s) only, and return the fallback ValidationResult even when it is a terminal non-ok failure.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@apps/webapp/app/services/realtime/jwtAuth.server.ts` around lines 47 - 107,
Only attempt the revoked-key fallback when the initial validateJWT failure is a
signature-verification error (e.g., check primaryResult.code for the signature
error code(s) instead of any !result.ok) and, inside
validateAgainstRevokedApiKeys, return the first fallbackResult you get for a
matching signature whether or not it is ok (so terminal errors like
expired/claim-invalid from a revoked-key-signed token are surfaced). Concretely:
change the caller to call validateAgainstRevokedApiKeys(token, signingEnvId,
result) only when result.code indicates a signature verification failure, and
change validateAgainstRevokedApiKeys to return the fallbackResult immediately
when validateJWT(token, apiKey) returns (not only when fallbackResult.ok) so the
fallback’s error codes are preserved.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Outside diff comments:
In `@apps/webapp/app/services/realtime/jwtAuth.server.ts`:
- Around line 47-107: Only attempt the revoked-key fallback when the initial
validateJWT failure is a signature-verification error (e.g., check
primaryResult.code for the signature error code(s) instead of any !result.ok)
and, inside validateAgainstRevokedApiKeys, return the first fallbackResult you
get for a matching signature whether or not it is ok (so terminal errors like
expired/claim-invalid from a revoked-key-signed token are surfaced). Concretely:
change the caller to call validateAgainstRevokedApiKeys(token, signingEnvId,
result) only when result.code indicates a signature verification failure, and
change validateAgainstRevokedApiKeys to return the fallbackResult immediately
when validateJWT(token, apiKey) returns (not only when fallbackResult.ok) so the
fallback’s error codes are preserved.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 48eb604b-8684-48bb-8f3e-36df24865df1

📥 Commits

Reviewing files that changed from the base of the PR and between fefe61f and 7a6c83e.

📒 Files selected for processing (3)
  • .server-changes/api-key-rotation-pat-grace-period.md
  • apps/webapp/app/models/runtimeEnvironment.server.ts
  • apps/webapp/app/services/realtime/jwtAuth.server.ts
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (30)
  • GitHub Check: audit
  • GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (7, 8)
  • GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (8, 8)
  • GitHub Check: units / internal / 🧪 Unit Tests: Internal (5, 8)
  • GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (1, 8)
  • GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (6, 8)
  • GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (3, 8)
  • GitHub Check: units / internal / 🧪 Unit Tests: Internal (8, 8)
  • GitHub Check: units / internal / 🧪 Unit Tests: Internal (6, 8)
  • GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (4, 8)
  • GitHub Check: units / internal / 🧪 Unit Tests: Internal (3, 8)
  • GitHub Check: units / internal / 🧪 Unit Tests: Internal (4, 8)
  • GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (5, 8)
  • GitHub Check: units / internal / 🧪 Unit Tests: Internal (2, 8)
  • GitHub Check: units / internal / 🧪 Unit Tests: Internal (7, 8)
  • GitHub Check: units / internal / 🧪 Unit Tests: Internal (1, 8)
  • GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (2, 8)
  • GitHub Check: units / packages / 🧪 Unit Tests: Packages (1, 1)
  • GitHub Check: units / e2e-webapp / 🧪 E2E Tests: Webapp
  • GitHub Check: sdk-compat / Node.js 22.12 (ubuntu-latest)
  • GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - npm)
  • GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - pnpm)
  • GitHub Check: sdk-compat / Bun Runtime
  • GitHub Check: sdk-compat / Node.js 20.20 (ubuntu-latest)
  • GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - npm)
  • GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - pnpm)
  • GitHub Check: sdk-compat / Deno Runtime
  • GitHub Check: sdk-compat / Cloudflare Workers
  • GitHub Check: typecheck / typecheck
  • GitHub Check: Analyze (javascript-typescript)
🧰 Additional context used
📓 Path-based instructions (8)
**/*.{ts,tsx}

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

**/*.{ts,tsx}: Use types over interfaces for TypeScript
Avoid using enums; prefer string unions or const objects instead

Files:

  • apps/webapp/app/models/runtimeEnvironment.server.ts
  • apps/webapp/app/services/realtime/jwtAuth.server.ts
{packages/core,apps/webapp}/**/*.{ts,tsx}

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

Use zod for validation in packages/core and apps/webapp

Files:

  • apps/webapp/app/models/runtimeEnvironment.server.ts
  • apps/webapp/app/services/realtime/jwtAuth.server.ts
**/*.{ts,tsx,js,jsx}

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

Use function declarations instead of default exports

Add crumbs as you write code using // @Crumbs comments or `// `#region` `@crumbs blocks. These are temporary debug instrumentation and must be stripped using agentcrumbs strip before merge.

Files:

  • apps/webapp/app/models/runtimeEnvironment.server.ts
  • apps/webapp/app/services/realtime/jwtAuth.server.ts
**/*.ts

📄 CodeRabbit inference engine (.cursor/rules/otel-metrics.mdc)

**/*.ts: When creating or editing OTEL metrics (counters, histograms, gauges), ensure metric attributes have low cardinality by using only enums, booleans, bounded error codes, or bounded shard IDs
Do not use high-cardinality attributes in OTEL metrics such as UUIDs/IDs (envId, userId, runId, projectId, organizationId), unbounded integers (itemCount, batchSize, retryCount), timestamps (createdAt, startTime), or free-form strings (errorMessage, taskName, queueName)
When exporting OTEL metrics via OTLP to Prometheus, be aware that the exporter automatically adds unit suffixes to metric names (e.g., 'my_duration_ms' becomes 'my_duration_ms_milliseconds', 'my_counter' becomes 'my_counter_total'). Account for these transformations when writing Grafana dashboards or Prometheus queries

Files:

  • apps/webapp/app/models/runtimeEnvironment.server.ts
  • apps/webapp/app/services/realtime/jwtAuth.server.ts
**/*.{js,ts,jsx,tsx,json,md,yaml,yml}

📄 CodeRabbit inference engine (AGENTS.md)

Format code using Prettier before committing

Files:

  • apps/webapp/app/models/runtimeEnvironment.server.ts
  • apps/webapp/app/services/realtime/jwtAuth.server.ts
**/*.ts{,x}

📄 CodeRabbit inference engine (CLAUDE.md)

Always import from @trigger.dev/sdk when writing Trigger.dev tasks. Never use @trigger.dev/sdk/v3 or deprecated client.defineJob.

Files:

  • apps/webapp/app/models/runtimeEnvironment.server.ts
  • apps/webapp/app/services/realtime/jwtAuth.server.ts
apps/webapp/**/*.{ts,tsx}

📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)

apps/webapp/**/*.{ts,tsx}: Access environment variables through the env export of env.server.ts instead of directly accessing process.env
Use subpath exports from @trigger.dev/core package instead of importing from the root @trigger.dev/core path

Use named constants for sentinel/placeholder values (e.g. const UNSET_VALUE = '__unset__') instead of raw string literals scattered across comparisons

Files:

  • apps/webapp/app/models/runtimeEnvironment.server.ts
  • apps/webapp/app/services/realtime/jwtAuth.server.ts
apps/webapp/**/*.server.ts

📄 CodeRabbit inference engine (apps/webapp/CLAUDE.md)

apps/webapp/**/*.server.ts: Never use request.signal for detecting client disconnects. Use getRequestAbortSignal() from app/services/httpAsyncStorage.server.ts instead, which is wired directly to Express res.on('close') and fires reliably
Access environment variables via env export from app/env.server.ts. Never use process.env directly
Always use findFirst instead of findUnique in Prisma queries. findUnique has an implicit DataLoader that batches concurrent calls and has active bugs even in Prisma 6.x (uppercase UUIDs returning null, composite key SQL correctness issues, 5-10x worse performance). findFirst is never batched and avoids this entire class of issues

Files:

  • apps/webapp/app/models/runtimeEnvironment.server.ts
  • apps/webapp/app/services/realtime/jwtAuth.server.ts
🧠 Learnings (12)
📓 Common learnings
Learnt from: myftija
Repo: triggerdotdev/trigger.dev PR: 2391
File: apps/webapp/app/services/organizationAccessToken.server.ts:50-0
Timestamp: 2025-08-14T10:53:54.526Z
Learning: In the Trigger.dev codebase, token service functions (like revokePersonalAccessToken and revokeOrganizationAccessToken) don't include tenant scoping in their database queries. Instead, authorization and tenant scoping happens at a higher level in the authentication flow (typically in route handlers) before these service functions are called. This is a consistent pattern across both Personal Access Tokens (PATs) and Organization Access Tokens (OATs).
📚 Learning: 2025-08-14T10:53:54.526Z
Learnt from: myftija
Repo: triggerdotdev/trigger.dev PR: 2391
File: apps/webapp/app/services/organizationAccessToken.server.ts:50-0
Timestamp: 2025-08-14T10:53:54.526Z
Learning: In the Trigger.dev codebase, token service functions (like revokePersonalAccessToken and revokeOrganizationAccessToken) don't include tenant scoping in their database queries. Instead, authorization and tenant scoping happens at a higher level in the authentication flow (typically in route handlers) before these service functions are called. This is a consistent pattern across both Personal Access Tokens (PATs) and Organization Access Tokens (OATs).

Applied to files:

  • .server-changes/api-key-rotation-pat-grace-period.md
📚 Learning: 2026-04-15T15:39:31.575Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/webapp.mdc:0-0
Timestamp: 2026-04-15T15:39:31.575Z
Learning: Applies to apps/webapp/**/*.{ts,tsx} : Access environment variables through the `env` export of `env.server.ts` instead of directly accessing `process.env`

Applied to files:

  • apps/webapp/app/models/runtimeEnvironment.server.ts
📚 Learning: 2026-04-15T15:39:31.575Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/webapp.mdc:0-0
Timestamp: 2026-04-15T15:39:31.575Z
Learning: Applies to apps/webapp/**/*.test.{ts,tsx} : Do not import `env.server.ts` directly or indirectly into test files; instead pass environment-dependent values through options/parameters to make code testable

Applied to files:

  • apps/webapp/app/models/runtimeEnvironment.server.ts
📚 Learning: 2026-04-16T14:19:16.330Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: apps/webapp/CLAUDE.md:0-0
Timestamp: 2026-04-16T14:19:16.330Z
Learning: Applies to apps/webapp/**/*.server.ts : Access environment variables via `env` export from `app/env.server.ts`. Never use `process.env` directly

Applied to files:

  • apps/webapp/app/models/runtimeEnvironment.server.ts
📚 Learning: 2026-03-02T12:42:47.652Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: apps/supervisor/CLAUDE.md:0-0
Timestamp: 2026-03-02T12:42:47.652Z
Learning: Applies to apps/supervisor/src/env.ts : Environment configuration should be defined in `src/env.ts`

Applied to files:

  • apps/webapp/app/models/runtimeEnvironment.server.ts
📚 Learning: 2026-03-22T13:26:12.060Z
Learnt from: ericallam
Repo: triggerdotdev/trigger.dev PR: 3244
File: apps/webapp/app/components/code/TextEditor.tsx:81-86
Timestamp: 2026-03-22T13:26:12.060Z
Learning: In the triggerdotdev/trigger.dev codebase, do not flag `navigator.clipboard.writeText(...)` calls for `missing-await`/`unhandled-promise` issues. These clipboard writes are intentionally invoked without `await` and without `catch` handlers across the project; keep that behavior consistent when reviewing TypeScript/TSX files (e.g., usages like in `apps/webapp/app/components/code/TextEditor.tsx`).

Applied to files:

  • apps/webapp/app/models/runtimeEnvironment.server.ts
  • apps/webapp/app/services/realtime/jwtAuth.server.ts
📚 Learning: 2026-03-22T19:24:14.403Z
Learnt from: matt-aitken
Repo: triggerdotdev/trigger.dev PR: 3187
File: apps/webapp/app/v3/services/alerts/deliverErrorGroupAlert.server.ts:200-204
Timestamp: 2026-03-22T19:24:14.403Z
Learning: In the triggerdotdev/trigger.dev codebase, webhook URLs are not expected to contain embedded credentials/secrets (e.g., fields like `ProjectAlertWebhookProperties` should only hold credential-free webhook endpoints). During code review, if you see logging or inclusion of raw webhook URLs in error messages, do not automatically treat it as a credential-leak/secrets-in-logs issue by default—first verify the URL does not contain embedded credentials (for example, no username/password in the URL, no obvious secret/token query params or fragments). If the URL is credential-free per this project’s conventions, allow the logging.

Applied to files:

  • apps/webapp/app/models/runtimeEnvironment.server.ts
  • apps/webapp/app/services/realtime/jwtAuth.server.ts
📚 Learning: 2026-04-20T15:08:55.358Z
Learnt from: ericallam
Repo: triggerdotdev/trigger.dev PR: 3417
File: apps/webapp/app/services/sessionsReplicationService.server.ts:204-215
Timestamp: 2026-04-20T15:08:55.358Z
Learning: In `apps/webapp/app/services/sessionsReplicationService.server.ts` and `apps/webapp/app/services/runsReplicationService.server.ts`, the `getKey` function in `ConcurrentFlushScheduler` uses `${item.event}_${item.session.id}` / `${item.event}_${item.run.id}` respectively. This pattern is intentionally kept identical across both replication services for consistency. Any change to the deduplication key shape (e.g., keying solely by session/run id) must be applied to both services together, never to one service in isolation. Tracking as a cross-service follow-up.

Applied to files:

  • apps/webapp/app/services/realtime/jwtAuth.server.ts
📚 Learning: 2026-04-20T15:06:19.815Z
Learnt from: ericallam
Repo: triggerdotdev/trigger.dev PR: 3417
File: apps/webapp/app/routes/realtime.v1.sessions.$session.$io.ts:37-51
Timestamp: 2026-04-20T15:06:19.815Z
Learning: In `apps/webapp/app/routes/realtime.v1.sessions.$session.$io.ts` (and all session realtime read paths), `$replica` is intentionally used for the `resolveSessionByIdOrExternalId` call — including the `closedAt` guard in the PUT/initialize path. The project convention is to use `$replica` consistently across all session realtime routes. The race window (replica lag allowing a ghost-initialize after close) is accepted as not realistic in practice (clients follow the close API response; they do not race it). If replica lag ever causes issues, the mitigation is to revisit all realtime routes together, not to swap individual routes to `prisma`. Do not flag `$replica` usage in session realtime routes as a stale-read issue.

Applied to files:

  • apps/webapp/app/services/realtime/jwtAuth.server.ts
📚 Learning: 2026-03-02T12:43:17.177Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: internal-packages/database/CLAUDE.md:0-0
Timestamp: 2026-03-02T12:43:17.177Z
Learning: Applies to internal-packages/database/**/{app,src,webapp}/**/*.{ts,tsx,js,jsx} : Use `$replica` from `~/db.server` for read-heavy queries in the webapp instead of the primary database connection

Applied to files:

  • apps/webapp/app/services/realtime/jwtAuth.server.ts
📚 Learning: 2026-03-26T09:02:07.973Z
Learnt from: myftija
Repo: triggerdotdev/trigger.dev PR: 3274
File: apps/webapp/app/services/runsReplicationService.server.ts:922-924
Timestamp: 2026-03-26T09:02:07.973Z
Learning: When parsing Trigger.dev task run annotations in server-side services, keep `TaskRun.annotations` strictly conforming to the `RunAnnotations` schema from `trigger.dev/core/v3`. If the code already uses `RunAnnotations.safeParse` (e.g., in a `#parseAnnotations` helper), treat that as intentional/necessary for atomic, schema-accurate annotation handling. Do not recommend relaxing the annotation payload schema or using a permissive “passthrough” parse path, since the annotations are expected to be written atomically in one operation and should not contain partial/legacy payloads that would require a looser parser.

Applied to files:

  • apps/webapp/app/services/realtime/jwtAuth.server.ts
🔇 Additional comments (1)
apps/webapp/app/models/runtimeEnvironment.server.ts (1)

112-127: Nice, this keeps the fallback lookup scoped without adding another read.

Pulling parentEnvironment.id into the existing findEnvironmentById() payload is a clean way to support the revoked-key lookup while keeping the return shape narrow.

@ericallam ericallam merged commit 99dfee3 into main Apr 29, 2026
45 checks passed
@ericallam ericallam deleted the fix/api-key-rotation-pat-grace-period branch April 29, 2026 09:00
@github-actions github-actions Bot mentioned this pull request Apr 29, 2026
ericallam pushed a commit that referenced this pull request May 1, 2026
## Summary
8 new features, 18 improvements, 11 bug fixes.

## Breaking changes
- Add server-side deprecation gate for deploys from v3 CLI versions
(gated by `DEPRECATE_V3_CLI_DEPLOYS_ENABLED`). v4 CLI deploys are
unaffected.
([#3415](#3415))

## Improvements
- Add `--no-browser` flag to `init` and `login` to skip auto-opening the
browser during authentication. Also error loudly when `init` is run
without `--yes` under non-TTY stdin (previously default-and-exited
silently, leaving the project half-initialized). Both commands now show
an `Examples` section in `--help`.
([#3483](#3483))
- Add `isReplay` boolean to the run context (`ctx.run.isReplay`),
derived from the existing `replayedFromTaskRunFriendlyId` database
field. Defaults to `false` for backwards compatibility.
([#3454](#3454))
- Redact the `resolveWaitpoint` runtime log so it only emits `id` and
`type` instead of the full completed waitpoint. Previously the log
printed the entire waitpoint (including `output`) to stdout in
production runs, which could leak sensitive payloads. The value returned
by `wait.forToken()` is unchanged.
([#3490](#3490))
- Add `SessionId` friendly ID generator and schemas for the new durable
Session primitive. Exported from `@trigger.dev/core/v3/isomorphic`
alongside `RunId`, `BatchId`, etc. Ships the
`CreateSessionStreamWaitpoint` request/response schemas alongside the
main Session CRUD.
([#3417](#3417))
- Truncate large error stacks and messages to prevent OOM crashes. Stack
traces are capped at 50 frames (keeping top 5 + bottom 45 with an
omission notice), individual stack lines at 1024 chars, and error
messages at 1000 chars. Applied in parseError, sanitizeError, and OTel
span recording.
([#3405](#3405))

## Server changes

These changes affect the self-hosted Docker image and Trigger.dev Cloud:

- Add a "Back office" tab to `/admin` and a per-organization detail page
at `/admin/back-office/orgs/:orgId`. The first action available on that
page is editing the org's API rate limit: admins can save a
`tokenBucket` override (refill rate, interval, max tokens) and see a
plain-English preview of the resulting sustained rate and burst
allowance. Writes are audit-logged via the server logger.
([#3434](#3434))
- Optional `DEPLOY_REGISTRY_ECR_DEFAULT_REPOSITORY_POLICY` env var to
apply a default repository policy when the webapp creates new ECR repos
([#3467](#3467))
- Ship the Errors page to all users, with a polish + bug-fix pass:
pinned "No channel" item in the Slack alert channel picker,
viewer-timezone alert timestamps via Slack's `<!date^>` token, Activity
sparkline peak tooltip, centered loading spinner and bug-icon empty
state on the error detail page, ellipsis on the Configure alerts
trigger.
([#3477](#3477))
- Configure the set of machine presets to build boot snapshots for at
deploy time via `COMPUTE_TEMPLATE_MACHINE_PRESETS` (CSV of preset names,
default `small-1x`). Use `COMPUTE_TEMPLATE_MACHINE_PRESETS_REQUIRED`
(CSV, default = full PRESETS list) to scope which preset failures fail a
required-mode deploy. Optional preset failures are logged and don't
block the deploy.
([#3492](#3492))
- Regenerating a RuntimeEnvironment API key no longer invalidates the
previous key immediately. The old key is recorded in a new
`RevokedApiKey` table with a 24 hour grace window, and
`findEnvironmentByApiKey` falls back to it when the submitted key
doesn't match any live environment. The grace window can be ended early
(or extended) by updating `expiresAt` on the row.
([#3420](#3420))
- Add the `Session` primitive — a durable, task-bound, bidirectional I/O
channel that outlives a single run and acts as the run manager for
`chat.agent`. Ships the Postgres `Session` + `SessionRun` tables,
ClickHouse `sessions_v1` + replication service, the `sessions` JWT
scope, and the public CRUD + realtime routes (`/api/v1/sessions`,
`/realtime/v1/sessions/:session/:io`) including `end-and-continue` for
server-orchestrated run handoffs and session-stream waitpoints.
([#3417](#3417))
- Add `KUBERNETES_POD_DNS_NDOTS_OVERRIDE_ENABLED` flag (off by default)
that overrides the cluster default and sets `dnsConfig.options.ndots` on
runner pods (defaulting to 2, configurable via
`KUBERNETES_POD_DNS_NDOTS`). Kubernetes defaults pods to `ndots: 5`, so
any name with fewer than 5 dots — including typical external domains
like `api.example.com` — is first walked through every entry in the
cluster search list (`<ns>.svc.cluster.local`, `svc.cluster.local`,
`cluster.local`) before being tried as-is, turning one resolution into
4+ CoreDNS queries (×2 with A+AAAA). Using a lower `ndots` value reduces
DNS query amplification in the `cluster.local` zone.
  
Note: before enabling, make sure no code path relies on search-list
expansion for names with dots ≥ the configured value — those names will
hit their as-is form first and could resolve externally before falling
back to the cluster search path.
([#3441](#3441))
- Vercel integration option to disable auto promotions
([#3376](#3376))
- Make it clear in the admin that feature flags are global and should
rarely be changed.
([#3408](#3408))
- Admin worker groups API: add GET loader and expose more fields on
POST. ([#3390](#3390))
- Add 60s fresh / 60s stale SWR cache to `getEntitlement` in
`platform.v3.server.ts`. Eliminates a synchronous billing-service HTTP
round trip on every trigger. Reuses the existing `platformCache` (LRU
memory + Redis) pattern already used for `limits` and `usage`. Cache key
is `${orgId}`. Errors return a permissive `{ hasAccess: true }` fallback
(existing behavior) and are also cached to prevent thundering-herd on
billing outages.
([#3388](#3388))
- Show a `MicroVM` badge next to the region name on the regions page.
([#3407](#3407))
- Increase default maximum project count per organization from 10 to 25
([#3409](#3409))
- Merge execution snapshot creation into the dequeue taskRun.update
transaction, reducing 2 DB commits to 1 per dequeue operation
([#3395](#3395))
- Add per-worker Node.js heap metrics to the OTel meter —
`nodejs.memory.heap.used`, `nodejs.memory.heap.total`,
`nodejs.memory.heap.limit`, `nodejs.memory.external`,
`nodejs.memory.array_buffers`, `nodejs.memory.rss`. Host-metrics only
publishes RSS, which overstates V8 heap by the external + native
footprint; these give direct heap visibility per cluster worker so
`NODE_MAX_OLD_SPACE_SIZE` can be sized against observed heap peaks
rather than RSS.
([#3437](#3437))
- Tag Prisma spans with `db.datasource: "writer" | "replica"` so
monitors and trace queries can distinguish the writer pool from the
replica pool. Applies to all `prisma:engine:*` spans (including
`prisma:engine:connection` used by the connection-pool monitors) and the
outer `prisma:client:operation` span.
([#3422](#3422))
- Clarify the cross-region intent in the Terraform and AI-prompt helpers
on the Add Private Connection page. Both already default
`supported_regions` to `["us-east-1", "eu-central-1"]`; added an inline
comment / parenthetical so the user understands why both regions are
listed (Trigger.dev runs in both, so the service must be consumable from
either).
([#3465](#3465))
- Add `RUN_ENGINE_READ_REPLICA_SNAPSHOTS_SINCE_ENABLED` flag (default
off) to route the Prisma reads inside `RunEngine.getSnapshotsSince`
through the read-only replica client. Offloads the snapshot polling
queries (fired by every running task runner) from the primary. When
disabled, behavior is unchanged.
([#3423](#3423))
- Stop creating TaskRunTag records and _TaskRunToTaskRunTag join table
entries during task triggering. The denormalized runTags string array on
TaskRun already stores tag names, making the M2M relation redundant
write overhead.
([#3369](#3369))
- Stop writing per-tick state (`lastScheduledTimestamp`,
`nextScheduledTimestamp`, `lastRunTriggeredAt`) on `TaskSchedule` and
`TaskScheduleInstance`. The schedule engine now carries the previous
fire time forward via the worker queue payload, eliminating ~270K
dead-tuple-driven autovacuums per year on these hot tables and the
associated `IO:XactSync` mini-spikes on the writer. Customer-facing
`payload.lastTimestamp` semantics are unchanged.
([#3476](#3476))
- Replace the expensive DISTINCT query for task filter dropdowns with a
dedicated TaskIdentifier registry table backed by Redis. Environments
migrate automatically on their next deploy, with a transparent fallback
to the legacy query for unmigrated environments. Also fixes duplicate
dropdown entries when a task changes trigger source, and adds
active/archived grouping for removed tasks. Moves BackgroundWorkerTask
reads in the trigger hot path to the read replica.
([#3368](#3368))
- Public Access Tokens (PATs) minted before an API key rotation now keep
working during the 24h grace window. `validatePublicJwtKey` falls back
to any non-expired `RevokedApiKey` rows for the signing environment when
the primary signature check against the env's current `apiKey` fails.
The fallback query only runs on the failure path, so the hot success
path is unchanged.
([#3464](#3464))
- Batch items that hit the environment queue size limit now fast-fail
without
retries and without creating pre-failed TaskRuns.
([#3352](#3352))
- Show the cancel button in the runs list for runs in `DEQUEUED` status.
`DEQUEUED` was missing from `NON_FINAL_RUN_STATUSES` so the list hid the
button even though the single run page allowed it.
([#3421](#3421))
- Reduce 5xx feedback loops on hot debounce keys by quantizing
`delayUntil`,
  adding an unlocked fast-path skip, and gracefully handling redlock
contention in `handleDebounce` so the SDK no longer retries into a herd.
([#3453](#3453))
- Fix RSS memory leak in the realtime proxy routes. `/realtime/v1/runs`,
`/realtime/v1/runs/:id`, and `/realtime/v1/batches/:id` called `fetch()`
into Electric with no abort signal, so when a client disconnected mid
long-poll, undici kept the upstream socket open and buffered response
chunks that would never be consumed — retained only in RSS, invisible to
V8 heap tooling. Thread `getRequestAbortSignal()` through
`RealtimeClient.streamRun/streamRuns/streamBatch` to `longPollingFetch`
and cancel the upstream body in the error path. Isolated reproducer
showed ~44 KB retained per leaked request; signal propagation releases
it cleanly.
([#3442](#3442))
- Fix memory leak where every aborted SSE connection pinned the full
request/response graph on Node 20, caused by `AbortSignal.any()` in
`sse.ts` retaining its source signals indefinitely (see
nodejs/node#54614, nodejs/node#55351). Also clear the
`setTimeout(abort)` timer in `entry.server.tsx` so successful HTML
renders don't pin the React tree for 30s per request.
([#3430](#3430))
- Preserve filters on the queues page when submitting modal actions.
([#3471](#3471))
- Fix Redis connection leak in realtime streams and broken abort signal
propagation.
  
**Redis connections**: Non-blocking methods (ingestData, appendPart,
getLastChunkIndex) now share a single Redis connection instead of
creating one per request. streamResponse still uses dedicated
connections (required for XREAD BLOCK) but now tears them down
immediately via disconnect() instead of graceful quit(), with a 15s
inactivity fallback.
  
**Abort signal**: request.signal is broken in Remix/Express due to a
Node.js undici GC bug (nodejs/node#55428) that severs the signal chain
when Remix clones the Request internally. Added getRequestAbortSignal()
wired to Express res.on("close") via httpAsyncStorage, which fires
reliably on client disconnect. All SSE/streaming routes updated to use
it. ([#3399](#3399))
- Prevent dashboard crash (React error #31) when span accessory item
text is not a string. Filters out malformed accessory items in
SpanCodePathAccessory instead of passing objects to React as children.
([#3400](#3400))
- Upgrade Remix packages from 2.1.0 to 2.17.4 to address security
vulnerabilities in React Router
([#3372](#3372))
- Fix Vercel integration settings page (remove redundant section
toggles) and improve the Vercel onboarding flow so the modal closes
after connecting a GitHub repo and the marketplace `next` URL is
preserved across the GitHub app install redirect.
([#3424](#3424))

<details>
<summary>Raw changeset output</summary>

# Releases
## @trigger.dev/build@4.4.5

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.5`

## trigger.dev@4.4.5

### Patch Changes

- Add `--no-browser` flag to `init` and `login` to skip auto-opening the
browser during authentication. Also error loudly when `init` is run
without `--yes` under non-TTY stdin (previously default-and-exited
silently, leaving the project half-initialized). Both commands now show
an `Examples` section in `--help`.
([#3483](#3483))
-   Updated dependencies:
    -   `@trigger.dev/core@4.4.5`
    -   `@trigger.dev/build@4.4.5`
    -   `@trigger.dev/schema-to-json@4.4.5`

## @trigger.dev/core@4.4.5

### Patch Changes

- Add `isReplay` boolean to the run context (`ctx.run.isReplay`),
derived from the existing `replayedFromTaskRunFriendlyId` database
field. Defaults to `false` for backwards compatibility.
([#3454](#3454))
- Redact the `resolveWaitpoint` runtime log so it only emits `id` and
`type` instead of the full completed waitpoint. Previously the log
printed the entire waitpoint (including `output`) to stdout in
production runs, which could leak sensitive payloads. The value returned
by `wait.forToken()` is unchanged.
([#3490](#3490))
- Add `SessionId` friendly ID generator and schemas for the new durable
Session primitive. Exported from `@trigger.dev/core/v3/isomorphic`
alongside `RunId`, `BatchId`, etc. Ships the
`CreateSessionStreamWaitpoint` request/response schemas alongside the
main Session CRUD.
([#3417](#3417))
- Truncate large error stacks and messages to prevent OOM crashes. Stack
traces are capped at 50 frames (keeping top 5 + bottom 45 with an
omission notice), individual stack lines at 1024 chars, and error
messages at 1000 chars. Applied in parseError, sanitizeError, and OTel
span recording.
([#3405](#3405))

## @trigger.dev/python@4.4.5

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.5`
    -   `@trigger.dev/build@4.4.5`
    -   `@trigger.dev/sdk@4.4.5`

## @trigger.dev/react-hooks@4.4.5

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.5`

## @trigger.dev/redis-worker@4.4.5

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.5`

## @trigger.dev/rsc@4.4.5

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.5`

## @trigger.dev/schema-to-json@4.4.5

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.5`

## @trigger.dev/sdk@4.4.5

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.5`

</details>

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants